Characterizing the Representer Theorem
نویسندگان
چکیده
The representer theorem assures that kernel methods retain optimality under penalized empirical risk minimization. While a sufficient condition on the form of the regularizer guaranteeing the representer theorem has been known since the initial development of kernel methods, necessary conditions have only been investigated recently. In this paper we completely characterize the necessary and sufficient conditions on the regularizer that ensure the representer theorem holds. The results are surprisingly simple yet broaden the conditions where the representer theorem is known to hold. Extension to the matrix domain is also addressed.
منابع مشابه
A Unifying View of Representer Theorems
It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the representer theorem, has been widely studied and applied to machine learning problems. Analogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we p...
متن کاملThe representer theorem for Hilbert spaces: a necessary and sufficient condition
A family of regularization functionals is said to admit a linear representer theorem if every member of the family admits minimizers that lie in a fixed finite dimensional subspace. A recent characterization states that a general class of regularization functionals with differentiable regularizer admits a linear representer theorem if and only if the regularization term is a non-decreasing func...
متن کاملA representer theorem for deep kernel learning
In this paper we provide a representer theorem for a concatenation of (linear combinations of) kernel functions of reproducing kernel Hilbert spaces. This fundamental result serves as a first mathematical foundation for the analysis of machine learning algorithms based on compositions of functions. As a direct consequence of this new representer theorem, the corresponding infinite-dimensional m...
متن کاملA Generalized Representer Theorem
Wahba’s classical representer theorem states that the solutions of certain risk minimization problems involving an empirical risk term and a quadratic regularizer can be written as expansions in terms of the training examples. We generalize the theorem to a larger class of regularizers and empirical risk terms, and give a self-contained proof utilizing the feature space associated with a kernel...
متن کاملJoint Regularization
We present a principled method to combine kernels under joint regularization constraints. Central to our method is an extension of the representer theorem for handling multiple joint regularization constraints. Experimental evidence shows the feasibility of our approach.
متن کامل